Principles and Practical Applications of Benchmarking

SUMMARY: As the information that is collected from health care encounters becomes more available, managed care pharmacy will have greater insights on the impact of pharmaceutical policy on patient outcomes. Database studies provide valuable information that depicts actual health care consumption and provides a tool to help manage the health care benefit. As compared with clinical trials, one of the strengths of database studies is that they are nonintrusive to patients and providers. However, the integrity of the data and any subsequent analysis are dependent on accurate and consistent coding practices at the time of data entry into the system. This article describes the 6 main steps required to complete a database study.

1. Disclose the principal sources of funding in a manner that permits easy recognition by the reader.
2. Disclose the existence of all potential conflicts of interest among supplement contributors, including financial or personal bias.
3. Describe all drugs by generic name unless the use of the brand name is necessary to reduce the opportunity for confusion among readers. 4. Strive to report subjects of current interest to managed care pharmacists and other managed care professionals.

5.
Seek and publish content that does not duplicate content in the Journal of Managed Care Pharmacy. 6. Subject all supplements to expert peer review.

Table of Contents
Principles and Practical Applications of Benchmarking fforts to provide optimal care while controlling overall costs have led multiple health care disciplines to invest in outcomes evaluation and monitoring. The comparison of outcomes to internal and external measures is known as benchmarking. 1 Benchmarking, a well-known business tool, is a term used to describe ongoing value measurement in industry-specific best practices.
Benchmarking is recognized as an essential technique for achieving continuous improvements in processes, performance measures, and operating efficiencies. These measurements are important for knowing an organization' s current status in comparison with competition or national trends. Additionally, a benchmark serves as a foundation of knowledge about current medical practice patterns that may guide decision making about future endeavors. In a business environment such as managed care that has a fundamental need for controlling cost, benchmarking can identify areas for improvement that can lead to practice innovations necessary to survive. 2 Benchmarking is integral to organizational and management strategies whose ultimate goal is to improve, and not simply measure, performance. 3 Benchmarking data in managed care is usually provided by retrospective evaluation of medical and pharmacy claims. Data collection, data warehousing, and electronic records research are essential activities to the benchmarking process. These, when used in conjunction with evidence-based guidelines, represent important components of efforts to improve patient outcomes and control costs of medical care.
Managed care pharmacy has demonstrated an increasing need for evidence-based guidelines to implement best practices in www.amcp.org Vol. 11

Target Audience
Pharmacists and other health care practitioners in a managed care environment

Learning Objectives
Upon completion of this program, the participant will be better able to 1. define the concept of benchmarking, 2. describe the 6 steps of database studies, 3. list the strengths and weaknesses of retrospective analysis of a database, 4. explain the usefulness of database studies as applied to managed care pharmacy, and 5. describe how benchmarking can be used as a management tool in decision making for quality improvement and formulary management activities.  formulary management, disease management programs, drug-use reviews, and other such activities. Table 1 shows some examples of online evidence-based resources. Evidence-based guidelines help professionals and patients choose the best available health care treatment options. In an era of increasing health care costs, the combination of benchmarking and the utilization of evidencebased guidelines provide managed care pharmacists with tools to assess whether practioners and the health care system are providing quality care for their patients. Factors that have influenced the growth and development of database studies are: • the emergence of electronic records research, • advancements in technology and computer capabilities, • growth of medical research and outcomes-based clinical studies, • concern about the rising costs of health care with a perceived nominal improvements in health outcomes, • easy access to medical information via the Internet, and • increased utilization of health care resources by various patient populations.
Evidence-based guidelines are developed by systematically reviewing and appraising available literature from multiple medical literature databases. It is a structured, yet multifaceted process of assuring that recommendations are based on evidence instead of anecdotal experience or other unscientific methods. Parameters such as quality, reproducibility, comprehensiveness, and type of study (e.g., clinical trial, meta-analysis, or case study) are used to grade the levels of evidence. For economic and decision analyses, the Centre for Evidence-Based Medicine at Oxford University has grouped the various sources into 5 levels of evidence. 4 (Table 2) This supplement presents information that illustrates the relationship between evidence-based medicine and benchmarking and explains how a benchmarking analysis is a useful management and decision-making tool for managed care pharmacists. The first article explains how to conduct a simple database study, including the strengths and weaknesses of a retrospective analysis. Following is a discussion that presents database management and applications of benchmarking from the perspective of managed care. Finally, case studies are presented to illustrate the application of the benchmarking process and its usefulness to managed care pharmacy.

TABLE 2
Level of Evidence Description of Evidence 1 Systematic review (with homogeneity) of level 1 economic studies Analysis based on clinically sensible costs or alternatives; systematic review(s) of the evidence; and including multiway sensitivity analyses Absolute better-value or worse-value analyses 2 Systematic review (with homogeneity) of level >2 economic studies Analysis based on clinically sensible costs or alternatives; limited review(s) of the evidence, or single studies; and including multiway sensitivity analyses Audit or outcomes research 3 Systematic review (with homogeneity) of level 3 studies Analysis based on limited alternatives or costs, poor quality estimates of data, but including sensitivity analyses incorporating clinically sensible variations 4 Analysis with no sensitivity analysis 5 Expert opinion without explicit critical appraisal, or based on economic theory or "first principles" M anaged care pharmacy has used pharmacy administrative databases for drug utilization review purposes since online claims adjudication has been in existence. However, as the information collected from all health care encounters becomes integrated and more readily available, managed care pharmacy will have greater insights about pharmaceutical impact on total patient outcomes. The availability of large amounts of data in administrative and clinical information systems provides a rich environment for conducting outcomes analyses in well-defined patient populations. 1 These database studies provide "real-world" information with the nuances of clinical practice patterns. This article will present some of the considerations for retrospective analyses and discuss the steps for conducting a simple database study.

Types of Analysis
Retrospective data analyses contain meaningful information that provide insights into past medical activity of patients, current managed care operations, and guidance for future steps. Retrospective analyses of health care data are often focused on economic or clinical research questions. While some studies forecast the cost and therapeutic impacts of new medications, other studies prepare the scientific groundwork for the development of new health improvement initiatives. Database studies may encompass 2 separate periods of time or be designed to track trends over time. Some of the common types of studies include those listed in Table 1.

Sample Size
The size of the study sample is critical to producing meaningful results and to determining if differences exist between 2 or more groups. The expected difference between 2 groups (study effect) determines the sample size needed to detect a change between the groups at a specified level of probability. Sample size calculations www.amcp.org Vol. 11

Author
ABSTRACT SUMMARY: As the information that is collected from health care encounters becomes more available, managed care pharmacy will have greater insights on the impact of pharmaceutical policy on patient outcomes. Database studies provide valuable information that depicts actual health care consumption and provides a tool to help manage the health care benefit. As compared with clinical trials, one of the strengths of database studies is that they are nonintrusive to patients and providers. However, the integrity of the data and any subsequent analysis are dependent on accurate and consistent coding practices at the time of data entry into the system. This article describes the 6 main steps required to complete a database study.  are important to ensure that, if an effect deemed to be clinically important exists, then there is a high chance of it being detected, (i.e., that the analysis will be statistically significant). 2 Statistical power is the probability of getting a statistically significant result given that there is a real effect in the population being studied. If a particular test is not statistically significant, it is because there is no effect or because the study design makes it unlikely that a biologically real effect would be detected. Power analysis can distinguish between these alternatives and is therefore a critical component of designing experiments and testing results. 2 (Power depends on many factors such as the type of test, α-level, and effect size; the reader is referred to specific statistical references for more information.) The sample size must be large enough so that the detected change is of scientific significance as well as statistical significance. 3 A study that is too small will not clearly show clinical significance. However, if the study sample is too large, an effect of little clinical importance is statistically detectable. 3

Strengths of Database Studies
Database studies offer several advantages over clinical trials. Most notably, database studies contain valuable information that depicts actual health care resource consumption. Practice and prescribing patterns can be observed without introducing extraneous variables. As database studies are always retrospective, there is never an alteration in individuals' behavior because of awareness of being studied or an increased awareness of their own behavior. Overall, database studies are nonintrusive to patients and providers, and data for large numbers of patients may be handled more quickly and less expensively than in a prospective clinical trial.
Results of retrospective studies are not usually subject to selection bias, which is a systematic error in creating intervention groups. Although patients are included or excluded based on several criteria, the criteria are applied to all patients in the database instead of first selecting the patients, then applying the treatment as in a prospective clinical trial. However, selection bias may be introduced by the inclusion and exclusion criteria used in the study design and whether or not incomplete records are used in the evaluation. For example, a patient who has depression and is not coded as having depression would not be included in a depressiondatabase study. Selection bias may also mean that the participants are not representative of the population of all possible participants.

Study Limitations
Certain aspects of database analysis also provide some challenges to the scientist or health care professional. Factors that pose the greatest challenges involve data collection and storage, areas where the analyst has little or no control. Significant variation in the accuracy and consistency of coding practices and associated data quality present a problem to the database researcher because if there is inconsistency in coding, there is inconsistency in the resulting judgments derived from that data. 4 Accurate data collec-tion relies on input ranging from proper ICD-9 (International Classification of Diseases, 9th edition) and procedure coding to the level of the retail pharmacy, where most outpatient pharmacy claims are generated.
For example, the pharmacy field "days supply" may be used as an indication of patient compliance. However, this field is often an inaccurate marker due to factors such as dosage titration schedules, unknown actual use, and medications used only as needed. Since direct communication with a patient is not practical, when the retrospective analysis shows, for example, potential noncompliance, there is no way to determine the reasons for the observed underutilization in medication therapy.
Database analyses require certain assumptions to be made, which are usually based on clinical knowledge about disease treatments, disease severity, and indicated and typical uses of medications. For example, alterations in practice patterns, such as off-label prescribing for a newly approved medication, need to be taken into consideration. To enable some reasonable assumptions to be made regarding patient types, medication therapy, or disease severity, a workable understanding of the disease being treated is important for effective database research. In the case of asthma, for instance, knowledge about types of inhalers used, usual inhaler use, and their relationship to disease severity will help the researcher to determine controlled or uncontrolled disease.
Other challenges to database studies that arise frequently are the age of the data and interdatabase compatibility. A simple example is a database of pharmacy claims generated from online pharmacy adjudication systems, which is generally available within 1 month. However, when the payer uses multiple vendors for pharmacy processing, or when mail order is required for chronic medications and local retail pharmacy for acute medications, multiple databases will need to be used to complete the study or important information will be missed. Other sources of medical data are somewhat older due to a delay in the claim submission process and are not immediately available for analysis. Many times, integrated databases require reprogramming into a compatible format to be usable. A data set may also need to be cleaned for duplicate records or missing values. These processes add to the age of the data, possibly up to 6 months.
As retrospective databases pose several methodological challenges, it should be noted that a checklist was recently published that can be used by decision makers to evaluate the quality of published studies that use health-related retrospective databases. 5

Database Variables
Variables in a database are pieces of information that can be specified based on the requirements of the study, such as the inclusion and exclusion criteria, patient groups, and the timeline of the study. Variables serve to limit the scope of the study, design the study, and group together information about the patients. They may be simple, such as inclusion criteria for age or insurance coverage, or complex, such as criteria for medication switching Essential Steps and Practical Applications for Database Studies within a certain timeline.
The goal of the study will help to determine some of the variables, such as the timeline of data used in the analysis. If the goal of a health plan' s study is to use the results to change patient or prescribing behavior, then a 3-month review of the data may be sufficient, whereas a longer history will produce more accurate results for an analysis of, for instance, morbidity or mortality outcomes associated with a particular intervention.

Proxy Outcomes
Clinical outcomes may be difficult to determine in database studies, as an observational study does not prove a cause-andeffect relationship. Instead, a proxy outcome, such as a hospital admission or an emergency room visit, may be used. Additionally, if medical data are not available, which is the case in most pharmacy benefit management databases, then the addition of a medication to a treatment regimen or medication usage patterns may act as the proxy outcomes.

ss Six Steps to Designing a Database Study
There are 6 main steps for completing a database study ( Table 2). 1 A brief discussion of each of the steps follows.

Step 1: Define the Study Objective
The purpose of the study, or the study objective, identifies the goals of the study and may be thought of with a statement such as, "I want to find out X from this database study." The study objective may be a simple cost assessment, conducted to make an administrative decision, or it might be a comparative pharmacoeconomic outcomes study, designed for external presentation and publication. This initial step may include a quick check to determine the capabilities of the database (e.g., the available data fields) and to ensure that the study design and conclusions are consistent with the database, although this is not considered a formal requirement of the first step. 6 Step 2: Identify the Data Elements Data elements are extracted from the database to help define the patient groups or the other units of comparison. Some examples of data elements are provided in Table 3. In comparison with database variables, which may be modified secondary to the requirements of the study (step 5 provides more information on this concept), the data elements cannot be changed. For ease of conducting the study, it is recommended to define the data elements as clearly as possible.

Step 3: Identify and Apply Specific Inclusion/Exclusion Criteria
To select a subset of eligible patients for further analysis, inclusion/exclusion criteria should be applied after the data have been extracted. The criteria must be consistent with the structure of the database elements. That is, the data elements may include a procedure code for a blood draw but not include the results of the blood draw. Therefore, the criteria cannot be dependent on specific lab results, only if a lab was taken. Each criterion should be applied in a stepwise manner, one criterion at a time, with the results of each application examined so that any adjustments can be made if unexpected results occur.
Database studies require that control groups be created, since patients often receive more than one type of treatment or have more than one diagnosis of interest. If costs and outcomes are to be compared for different groups of patients, the comparison groups need to be created using key data elements to identify the appropriate patients. Control groups must be well matched for disease, diagnoses, comorbid conditions, age, and gender. Differences in patient characteristics (e.g., age, comorbidities, and severity of illness), pharmaceutical therapy, and other important clinical differences between the groups may then be summarized and reported.

Step 4: Perform Initial Data Analyses and Review Preliminary Results
Initial analyses should be conducted to ensure that sufficient numbers of patients still exist after applying the restriction criteria conforming to the predefined power analysis. Data should be summarized for variables that are easy to analyze, such as the number and percentage of patients who meet the inclusion criteria, have the diagnosis of interest, or used the prescribed medications of interest. Other information, such as patient characteristics and patient counts, should also be reviewed as part of the initial analysis. Steps to Design a Database Study   TABLE 2 1. Define the study objective 2. Extract key data elements 3. Apply inclusion, exclusion criteria 4. Perform initial data analyses 5. Create "calculated" analysis variables 6. Compare groups

Example of Resulting Patient Counts After an Initial Data Analysis and Then the Complex Data Analysis
Step 5: Create "Calculated" Analyses Variables Some complex analyses can be made easier by creating new calculated variables to facilitate data analysis and summarization. A common example is the calculation of patients' ages at a certain time by subtracting each person' s date of birth from their date of hospitalization. Another example might be the calculation of persistence by identifying refill dates and amount dispensed and averaging over a preidentified time period. These new calculated variables may be placed in a separate column for further analysis.
Step 6: Apply the Appropriate Statistical Tests Lastly, the appropriate statistical tests are applied to evaluate the significance of the differences. When reporting study results, it is important to recognize the difference between statistical and practical significance of the findings. 6 A study may show statistical significance, but it is always prudent to ask "What are the practical outcomes of this study?" as it relates to the population in question.

Applications
While one or two database studies provide a snapshot in time, a series of database study results provide a tool to track trends over time. If members are enrolled with the same managed care plan over time, long-term costs and clinical outcomes for chronic diseases, and the impact of different factors (e.g., pharmacotherapy, physician specialties) on disease outcomes may become apparent. Table 4 provides some managed care applications of database studies.
A series of results allows a greater understanding of the overall market, including topics such as how medications are used to treat a disease, markers of concomitant medication use, patient compliance with prescribed therapy, typical morbidity markers for the disease, and concomitant disease processes. A comprehensive understanding of the market may help to predict the economic impact of a newly approved medication whether it is from the perspective of the clinical pharmacist, the president of the managed care plan, or the pharmaceutical manufacturer.

ss Summary
Managed care pharmacy has used administrative databases for drug utilization review purposes since online adjudication has been in existence. As the information collected from health care encounters becomes more readily available, managed care pharmacy will have ever greater insights about pharmaceutical impact on total patient outcomes. With limited resources, database research in health plans is conducted to provide information that may help to shape cost-saving measures or to improve the care or delivery of care for patients. Administrative databases are a useful source of data for retrospective studies that evaluate the effects of policy change on new programs and pharmaceutical therapy. Since administrative databases were not designed for research purposes, care must be exercised to overcome their limitations and to ensure credible results when outcomes for different groups of patients are being compared.   • Identify patient populations that may be targeted for patient educational programs • Identify therapeutic areas where patients and the health plan would benefit from prospective clinical programs • Manage areas of high cost • Identify prescribing patterns that fall outside of best practices • Alert a prescriber of patient behavior (e.g., controlled substance drug utilization review) M anaged care has come a long way from the days of simple cost consolidation and cooperative patient care. Whereas traditional managed care business activities, such as provider contracting and provider networking, continue to be core elements in managed care design, the entire business environment has expanded. Coverage now commonly includes diagnostic screening, chiropractic services, and disease management initiatives, while some measures, such as multi-tiered copayment structures and prior authorization, are implemented to deter all-encompassing use of health care. Taken together, these activities form the current state of managed care' s efforts to control costs while maintaining or improving the health of its members.

Some Examples of Data Elements
Evidence-based disease management programs have played an important role in efforts to improve the health of patients who have chronic diseases. Efforts have been directed toward both patients and physicians, and the methods used to reach these audiences have ranged from sponsorship of Internet sites to targeted mailings. Intensive case management of high-risk and high-cost patient groups may be contracted with outside vendors to streamline care and reduce costs. 1 The development and execution of these and other clinical programs allows health plans to work toward the goal of optimizing members' health while controlling total health care costs.

ss Managed Care and Benchmarking
Measurement of the outcomes associated with implementation of evidence-based practice programs is becoming increasingly emphasized by multiple health care disciplines. 2 As previously stated, the comparison of outcomes to internal and external measures is known as benchmarking. 2 Benchmarking reaches beyond typical retrospective analyses to include fields such as physician specialties, comprehensive medication usage, and concomitant diseases. The entire dataset is systematically organized so that it can focus on and compare various measures such as treatment patterns, age groups, patient groups, physician specialty, and regional differences. The list of analyses is endless, as there are no consistent measures.

ss Managed Care and Database Management
In the early days of disease management, many programs were implemented based on the knowledge that poor health leads to more expensive care. Unfortunately, there was scant evidence to support the idea that a particular program would improve health and decrease costs. Additionally, pharmacoeconomic studies proving a cost benefit after program implementation were lacking, and there was an ever-present inverse relationship-and conflict-between the medical and pharmacy sides of the cost equation. Since then, evidence-based medicine has become more accepted and, in some areas, required; pharmacoeconomic data are somewhat easier to come by; and disease management programs have forged ahead to include areas such as seizure disorders, rheumatoid arthritis, and multiple sclerosis. 1 Concurrently, managed care has realized multifactorial value to investing in clinical care-management programs. Besides benefits of improved member health, the member' s participation in the clinical program generates data, which are captured and stored for subsequent analysis. The data that a health plan collects may be used for internal retrospective data analyses to facilitate drug use reviews, targeted patient or physician mailings, internal benchmarking, and client benchmarking at a regional or national level. With a truly integrated medical and pharmacy database, benchmarking could become a valuable tool on many fronts.

ss Usefulness of Benchmarking to Managed Care Pharmacy
Benchmarking data offer several advantages to managed care and managed care pharmacy. Primarily, the information gleaned from the benchmarking dataset contributes to enhanced medical knowledge and can be used to improve patient care or implement cost-savings programs. Depending on how the data are analyzed, the benchmarking dataset can provide a current market analysis, show where there are gaps in disease management offerings, or provide insights into the need to optimize therapy or how a formulary decision may impact the cost of care or types of care given.
As medication formularies have undergone significant transformation in the past 8 to 10 years, the impact of benchmarking on formulary decisions demonstrates the usefulness of this type of analysis to managed care. Whereas, in the past, open formularies covered most prescription medications, within the last few years, resource constraints have necessitated tightened coverage for prescription medications as well as implementation of tiered copayments, quantity-level limits, mandatory generics, or exclusion of certain classes all together (i.e., medications for allergies). Decision makers have had to consider coverage issues ranging from topics such as recent prescription to over-the-counter switches (loratadine [Claritin], omeprazole [Prilosec]) to the expanding use of expensive biologics, antitumor medications, and other injectables.
Benchmarking helps decision makers make formulary management decisions based on knowledge about the environment in which the medication will be used and prescribed. Benchmarking offers insights into how a disease is treated in a real-world setting and therefore allows some assumptions to be made regarding use or misuse of a medication within a defined patient population. For example, if the benchmarking data show that patients who are treated with medication XYZ have reduced visits to other physician specialties, then the decision for formulary inclusion of that medication can be based on some knowledge of the cost benefit of that medication. Conversely, if the data show that patients who have asthma are optimally controlled when they are compliant with medications already on the market, then it could be justified to exclude from coverage a new, yet highly priced, asthma medication, where limited or no benefit is realized from the increased expenditure.
Benchmarking data may also be used as tools that help to determine where limited resources for disease management programs should be directed. Analyses of the patient populations and the costs related to their respective disease states give a managed care organization the ability to invest in disease areas where clinical and cost benefits are more certain. Health care expenditures are a classic application of the 80/20 rule (i.e., 80% of the total cost is generated by 20% of the population.). By understanding which types of patients consume the most resources as well as using evidence-based guidelines as a clinical basis for improved patient care, efforts can be focused on areas that will have the most impact. How the impact is measured, of course, is another topic, and is determined by the health plan.
Benchmarking may also be used to perform regional comparisons that help a health plan determine internal performance versus the competition or against national trends. Similar populations within different regions of the state or nation may be compared with respect to medication-cost trends, chronic disease management, procedures, or types of therapy for certain diseases. For example, Krumholz and colleagues performed an evaluation of patients in New England who were hospitalized with myocardial infarction (MI) (1994-1996) to assess regional variations in quality of care. 4 Their results showed that compared with patients in other U.S. regions, post-myocardial infarction patients in New England had higher rates of indicated pharmacotherapy use, a lower use of reperfusion therapy, and the lowest risk-standardized 30-day mortality rate (all measures were significant). In a retrospective study of only pharmacy claims, Cox and colleagues demonstrated that there are geographic variations in the prevalence of stimulant use among patients aged 5 to 14 years. 5 Although the results of these types of studies raise questions for future research, they also raise awareness within managed care or other health care organizations to the need for frequent internal analysis of the optimal use of the health care services available.
Comparing value across programs may provide more accurate assessments of performance, enhance quality improvement efforts, and contribute generalizable knowledge. 6 While benchmarking can lead to practice innovations necessary for survival in an environment that has a need for decreasing cost and increasing quality, it can be used in other areas. 7 For instance, if the data show that an organization is meeting national care guidelines or leading the industry in a certain area, it allows marketing claims based on fact, which is a very strong factor in advertising.
Benchmarking also helps to answer several questions. Some examples include: 1. What is occurring in clinical practice? 2. How is a medication or medical product being used? 3. What type of population is using it? 4. What other medications are these patients using? 5 have not yet received the program serve as the comparison group. • Regression-discontinuation design. This evaluation design can be used when a program or some of its components are delivered to high-risk persons with an index condition. The data analysis takes advantage of the correlation of preintervention and postintervention metric scores among intervened and nonintervened persons.
ss Summary Health plans, once simply thought of as "the payer" or "the insurance company," are now more involved in maintaining or improving the health of their patient populations. The focus of their business has changed to include not only the responsibility of making ill people well but also of trying to keep people healthy. Managed care organizations have realized multiple benefits from investing in clinical program efforts, mainly improved patient health, and the availability of large amounts of health care data. Health plans are able to collect, analyze, and distribute information on a variety of chronic conditions, acute illnesses, and treatments. Data management is an integral part of health plan information technology. Data that a health plan collects usually are kept proprietary and may be used for internal retrospective data analyses to facilitate drug utilization reviews, targeted patient or physician mailings, internal benchmarking, or client benchmarking at a regional or national level. Benchmarking offers a method of establishing standards for use with clinical governance. Benchmarking data are a powerful tool for health plan decision makers because they offer insights into how decisions about medication selection, formulary management, quality improvement initiatives, and other health plan activities may affect future operations of the company. L arge investments have been made in health care data warehousing and data analysis over the past 10 years. Health plans, pharmacy benefit management companies, employers, and other health care entities are increasingly interested in the ability to integrate medical and pharmacy claims data to provide even more valuable information than either dataset provides by itself. Data integration has allowed health care entities to perform robust clinical analyses as well as cost-benefit investigations in areas that were previously very difficult to research because of the separate storage for medical and pharmacy claims.
Hosts of consulting organizations have emerged to analyze the data generated by managed care organizations and provide services such as outcomes and clinical research, strategy, and market analysis. Generally, these consultant companies compile outcomes solely from internal analysis of the managed care organization' s internal data, and the results are provided only to the managed care organization.
In contrast, a particularly large dataset from Pharmetrics, Inc. is being combined with disease benchmarking technology by The MCM Group, a health care marketing and communications company. The dataset is an integrated health care database (linking medical and pharmacy claims) of millions of lives organized into disease states and linked to all of the services and costs that have been captured for the patients with that disease. (  database, and Episode Treatment Groups (ETGs) to facilitate analysis of inpatient, outpatient, ancillary services, and pharmaceutical utilization and costs. These patient-specific utilization and cost parameters are matched to the diagnosis and linked to an episode of care. As a result of the use of ETGs, all costs, including pharmaceuticals, can be measured, tied to a diagnosis, and compared with other treatment options. Three years of the most recent and available raw claims data are extracted from an independent third-party database and condensed into a usable form (see Tables 1 and 2).
The Disease Benchmarks program was developed to create opportunities for health care decision makers to evaluate the entire health care continuum in a disease-specific fashion. It is designed to help health plans manage the provision of care as well as the cost of care. The data that is presented includes all costs, units of use, and services associated with treating a specific disease and is reported by category (Inpatient, Outpatient, Emergency Department, and Pharmacy). The costs depicted in the benchmarking publications are direct charges back to the system, that is, the actual charged cost of providing that care. Additionally, a paid-to-charged ratio is provided for each disease if users are interested in looking more specifically at actual "paid" data.
With the data and applications made available through Disease Benchmarks, health care organizations can compare their own data for a specific disease against independently generated national and regional benchmarks. The benchmarking program also allows health plans to better understand the impact that various treatment options and patterns, including pharmacotherapy, have on both clinical and economic outcomes. This provides an essential platform for more effective disease management.

ss Pharmaceutical Partnerships
Pharmaceutical manufacturers have partnered with The MCM Group to provide individual Disease Benchmarks to organized customers. The manufacturers have stated a commitment to developing a benchmarking program that provides a valuable resource for health care decision makers.
The MCM Group has forged several pharmaceutical partnerships to develop health economic studies involving organizations such as Aventis for diabetes; Amgen for anemia and rheumatoid arthritis; Solvay for irritable bowel syndrome and hypogonadism; AstraZeneca for hypertension, coronary artery disease, congestive heart failure, atrial fibrillation, and deep vein thrombosis; Genentech for asthma; Roche for hepatitis C; Serono for multiple sclerosis and infertility; Wyeth/Solvay for mental health benchmarks; and Daiichi Pharmaceuticals for otitis externa, acute otitis media with tympanostomy tubes (AOM/TT), and Sjögren' s syndrome.
For the purposes of this review, and to give some depth to the capabilities of the disease benchmarking process, otitis externa, AOM/TT, and Sjögren' s syndrome will serve as examples for case presentations. Three separate disease models were developed that condense data aggregated from more than 27 million unique patients and 53 health plans. Each disease model identifies eligible patients using a combination of ICD-9-CM (International Classification of Diseases, Ninth Revision, Clinical Modification) diagnosis codes present on medical claims and the proprietary episode-creating software system, ETGs.
Factors believed to impact the cost of care are used to stratify and segment eligible patients into homogeneous groups for collecting and reporting data. Some of the factors include year, geography, age, gender, pharmacotherapy, comorbidities and complications, and treatment by physician specialty.

ss Value of Disease Benchmarking
The Benchmarks program is valuable because of its flexibility and because it depicts what is occurring in clinical practice. Of course, as in any retrospective analysis, direct clinical conclusions cannot be made. The Benchmarks program makes use of many different variables to determine which factors impact cost and utilization. It can be customized to compare different age groups, care by physician specialty, treatment by gender, or regional variations in treatment. More specific information can also be obtained, such as determination of units of use (the number of units of the actual health care deliverable used), the number of prescriptions used for a particular disease, which drug class was used, and the number of prescriptions per episode. The number of physician visits for an episode, and the number and type of specialist versus primary care physicians seen are other examples of data that can be retrieved via the disease model.

ss Bringing It to a Practical Level
The case presentations below show some of the practical applications of the Benchmarks program. A brief clinical background of the disease is followed by the patient selection criteria and then some of the pharmacy findings. These findings highlight the capabilities of Disease Benchmarks. For example, in the otitis externa case presentation, costs are separated by patient group and Episode treatment groups are an illness classification system matching claims to disease episodes for measuring and comparing utilization and cost of health care management. The ETG methodology accounts for case-mix adjustment, clinical homogeneity, episode building, concurrent/recurrent episodes, and shifting episodes.
ETGs are used by more than 450 health care organizations nationally to facilitate the analysis of inpatient, outpatient, ancillary, and pharmaceutical costs. Through ETGs, patient-specific costs are matched to a patient-specific diagnosis and are dynamically linked to a specific episode of care to measure resource consumption.  reported by class of antibiotic. The AOM/TT case presentation demonstrates that patients are selected by the presence of diagnosis codes and CPT-4 (common procedural terminology) codes and are then followed over time for the occurrence of subsequent pharmacy or medical claims. The Sjögren' s syndrome case presentation provides information about the demographics of the patient population and 3-year total cost trends for each medication class. Each of these measures is available for all of the disease states, but these particular disease states provide excellent material for showcasing the potential of any well-designed benchmarking-type study.

Episode Treatment Groups (ETGs)
ss Case Presentations

Case Study 1: Otitis Externa Clinical Background
Otitis externa, 1 which is characterized primarily by inflammation of the external auditory canal, affects approximately 1 in 250 people in the United States each year. 2 Otitis externa is typically a bacterial and localized process that generally responds to treatment with topical antibiotic agents, but if it is not managed appropriately, it can invade the surrounding tissues and become a serious life-threatening condition. 3,4 In many cases, the associated otalgia is significant enough to disrupt daily functioning. The most common pathogen in otitis externa is Pseudomonas aeruginosa. Of these episodes, only those containing diagnosis codes for otitis externa (ICD-9-CM codes 380.1 or 380.2) were selected for analysis. This dataset included claims from calendar years 1999-2001.

Findings
Overall, most otitis externa episodes observed were 20.5 days in duration ( Table 3). The costs and units of use by episode type are analyzed into subtypes respective of the ETG groups for this disease. In otitis externa episodes without surgery, the presence of comorbidities had little impact on episode economics (Tables 4  and 5). Table 6 demonstrates the cost breakdown for otitis externa episodes without surgery by oral and otic antibiotic use.

Case Study 2: Acute Otitis Media with Tympanostomy Tubes Clinical Background
Otitis media 5 is a term used to characterize inflammation of the middle ear without reference to etiology or pathogenesis. Otitis

Applications of Disease Benchmarks and Case Presentations
media results most often from a dysfunction of the eustachian tube, which is responsible for ventilating and equalizing air pressure in the the middle ear cavity. 6 A short eustachian tube length may contribute to insufflation or reflux of the nasopharyngeal contents into the middle ear cavity. 7,8 When this occurs, the mucosal lining becomes inflamed, limiting the ventilatory capacity of the eustachian tube. 7,8 In addition, the mucosal lining begins to produce fluid, resulting in the accumulation of fluid in the middle ear cavity. 8 TTs may be used to restore ventilation to the middle ear cavity and to decrease the incidence of AOM in otitis-prone children. 8 However, otitis media recurrence after TT placement is characterized primarily by otorrhea, and is termed posttympanostomy tube otorrhea (PTTO). 9 PTTO is primarily attributed to a bacterial infection, and ototopical medications are considered initial treatment. 10 The antibiotic chosen should cover Pseudomonas aeruginosa and Staphylococcus species in addition to the usual pathogens seen in AOM (Streptococcus pneumoniae, Haemophilus influenzae, and Moraxella catarrhalis). 10,11 TTs permit direct antibiotic application to the middle ear cavity, allowing for high concentrations at the site of infection. Topical therapy has been shown to cause fewer systemic adverse effects than oral therapy. 9

Patient Selection
Patients were selected for analysis by the presence of ETG-defined episodes of care for AOM and the presence of TT placement services. Once unique patients were selected, the first AOM episode with TT placement services was identified by CPT-4 coding (codes 69433-69436). Moving forward from that point, all other AOM episodes were captured and analyzed and placed into the study period in which it occurred (i. e., 1999, 2000, or 2001). In all, there were 3 mutually exclusive and mutually exhaustive episode types observed in this analysis: • AOM with Primary TT Placement. These AOM episodes are the first episodes reported in a patient history with TT placement services. • AOM with Secondary TT Placement. These AOM episodes also contain TT placement services, and occur after an "AOM with Primary TT" episode. • Subsequent AOM. These AOM episodes also occur after an "AOM with Primary TT" episode, but do not contain any TT placement services.

Pharmacotherapy Segmentation
Episodes were categorized by the presence of claims for specific    drug classes. To be classified with the use of a drug class, at least 1 claim from a specific drug class (identified by national drug code) was required to be present within the episode of care at any time during the episode. The use of the following drug classes was observed:

Findings
Tables 7, 8, and 9 present some of the findings of the Disease Benchmarks Program for AOM/TT. Across all episode types and antibiotic therapy, the lowest total episode costs were consistently associated with the sole use of otic antibiotic therapy.

Case Study 3: Sjögren's syndrome 12 Clinical Background
Sjögren' s syndrome is a systemic autoimmune disease that is characterized by lymphocytic infiltration of exocrine glands, primarily lacrimal or salivary glands. [13][14][15][16] Chronic and episodic damage to the lacrimal and salivary glands results in a gradual loss of ability to secrete tears and saliva, respectively, leading to worsening dry eyes and dry mouth. Many patients also experience debilitating fatigue and joint pain. 17 The course of the disease is unpredictable, with symptoms that can flare or go into remission and range in intensity from mild to incapacitating. 18,19 Although the hallmark symptoms are dry eyes and dry mouth, other exocrine glands can be affected resulting in impaired functioning of the respiratory, gastrointestinal, gynecologic, skin, and hepatic systems. In most patients, the primary syndrome runs a slow and benign course. Eight to 10 years often elapse from the initial symptoms to full-blown development of the disease. 16,19 In this time, patients often present to multiple specialists, each of whom sees only a restricted part of the syndrome. 19 There are no formal analyses of the health care costs for patients with Sjögren' s syndrome. However, patients often spend years seeking an accurate diagnosis, partly because of the waxing and waning nature of the disease and partly because of the difficulties and lack of consensus in understanding, diagnosing, and treating Sjögren' s syndrome. Because of the concomitant disorders, lack of adequate treatments, and out-of-pocket patient expenses, it is likely a costly disease for patients who suffer from it.

Patient Selection
Patients were selected based on the presence of one or more medical service claims with ICD-9 diagnosis code 710.2 (sicca syndrome, Sjögren' s syndrome). Patients were separated into 1 of 2  Three-Year Cost Trends-Subsequent AOM Episodes*  groups: those with a definitive diagnosis for Sjögren' s syndrome (Sjögren' s population) and those who were determined to be at risk for developing this disease (at-risk population). Patients were selected as part of the Sjögren' s syndrome at-risk population based on the presence of one or more "risk factor" conditions (i.e., systemic lupus erythematosus, rheumatoid arthritis, etc.), but without the presence of diagnosis codes for Sjögren' s syndrome. Each episode consists of observations made over an interval of 365 days. This dataset included claims from calendar years 1999-2001.

Findings
There was a consistent distribution of data from the studied years, 1999 through 2001, with respect to the categories of average age, percentage of patients in each age group, and percentage of female and male patients. Here are some of the findings: • Patients with Sjögren' s syndrome were, on average, 52.4 years of age, and 88.3% of the total population was female (Table 10). • Compared with internists and family/general practitioners, rheumatologists were more likely to prescribe cholinergicenhancing agents but less likely to prescribe antidepressants, oral antifungals, topical vaginal estrogens, and inhaled nasal steroids. • The average yearly cost to treat Sjögren' s syndrome was $1,938, with approximately 46% of the charges generated from outpatient services and 47% from pharmacy services. • Total costs of care were highest for the age group 36 to 49 years (Table 11). • In the "at-risk" population, there was a significant amount of Sjögren' s syndrome-related costs (especially for drugs common to Sjögren' s), but without a diagnosis for Sjögren' s syndrome, the average yearly cost was $1,208.85, with pharmacy services accounting for nearly 78% of the total annual costs. • The "at-risk" population used similar pharmacologic therapy as the Sjögren' s syndrome population, and the total units of use (i.e., prescriptions) per episode were comparable between groups (   being treated. In this era, when decisions about pharmacy and medical coverage-and limitations of coverage-are being made and patients are directly affected, the usefulness of this knowledge cannot be understated. It helps to alert decision makers to the real versus perceived "need" of some drugs, while it may also highlight whether a decision to limit coverage is realistic for a particular patient population. Internal financial projections based on certain summaries in the benchmarking dataset may help to form clinical programs or coverage policies. Finally, managed care organizations can use the data to understand the potential impact that formulary or coverage decisions may have on the affected population.
ss Summary In an environment that has a need for decreasing cost and increasing quality, benchmarking can lead to practice innovations necessary to survive. The TRU Disease Benchmarks program was developed to provide a better understanding of costs and services used in association with a particular disease. The outcomes data obtained from the benchmarking program is extremely flexible and provides insights into how the disease is treated in clinical practice, the cost associated with one episode, and which comorbidities and complications are linked with that particular disease. The real-world setting of the benchmarking database allows some assumptions to be made regarding use or misuse of a product within a defined patient population and therefore can be used as a management tool in decisions about medication coverage. It may help to answer questions such as: "How will a drug affect the pharmacy budget?" and "What might be the impact on the medical side?"  can be categorized by a. oral penicillin agents < otic quinolone solution < otic quinolone suspension < oral penicillin-combination agents b. otic aminoglycoside suspension < oral cephalosporins < oral penicillins c. otic quinolone suspension < otic quinolone solution < otic aminoglycoside suspension 8. The Disease Benchmarks Program for acute otitis media with tympanostomy tubes shows that, across all episode types and antibiotic therapy, the lowest total episode costs were consistently associated with the sole use of otic antibiotic therapy. a. True b. False 9. A comprehensive knowledge of the disease being treated is derived from benchmarking data that allows insights into a. how a disease is being treated, demographics of the patient population, and where resources are being used. b. evidence-based sources of information, talking with 5 people who suffer from the disease, and a survey of family physicians. c. where resources are being used, current practice patterns, and actual drug spend.

Continuing Education Principles and Practical Applications of Benchmarking
Please indicate the correct answers on the Record of Completion.
10. The delay in availability of medical claims generally adds integrity and value to the data set. a. True b. False 11. In comparison with database variables, which may be modified secondary to the requirements of the study, the data elements cannot be changed. a. True b. False 12. Managed care organizations use database studies in a variety of applications, which may include a. drug utilization reviews. b. identifying treatment patterns for a targeted population. c. financial projections of adding a clinical staff member. d. a and b e. a, b, and c 13. If a newly approved medication for hypertension is reviewed at the P&T meeting, evidence-based guidelines in combination with benchmarking data may allow P&T members insight that will affect a. how patients will tolerate the medication. b. formulary inclusion. c. coverage limitations. d. b and c e. a, b, and c 14. Regional or national comparisons of the relationship between treatment and outcomes raises awareness within managed care organizations of the need for a. drastic cuts in pharmacy coverage. b. massive expansion in prior-authorization criteria. c. frequent internal analysis of the optimal use of the health care services available.  The learning objective I was least able to achieve was: In this program, I would have liked to learn more about: Please describe one or more challenges you face in practice about which you wish to learn more.
Please list any other comments that you may have about this program. Please describe changes: